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Abstract — This paper considers the problem of computing 
the schedule of modes in a switched dynamical system, that 
minimizes a cost functional defined on the trajectory of the 
system's continuous state variable. A recent approach to such 
optimal control problems consists of algorithms that alternate 
between computing the optimal switching times between modes 
in a given sequence, and updating the mode-sequence by 
inserting to it a finite number of new modes. These algorithms 
have an inherent inefficiency due to their sparse update of 
the mode-sequences, while spending most of the computing 
times on optimizing with respect to the switching times for 
a given mode-sequence. This paper proposes an algorithm that 
operates directly in the schedule space without resorting to the 
timing optimization problem. It is based on the Armijo step 
size along certain Gateaux derivatives of the performance func- 
tional, thereby avoiding some of the computational difficulties 
associated with discrete scheduling parameters. Its convergence 
to local minima as well as its rate of convergence are proved, 
and a simulation example on a nonlinear system exhibits quite 
a fast convergence. 



I. Introduction 

Switched-mode hybrid dynamical systems often are char- 
acterized by the following equation, 



f(x,v) 



(1) 



where x £ R n is the state variable, v £ V with V being a 
given finite set, and / : R n x^-> R n is a suitable function. 
Suppose that the system evolves on a horizon-interval [0, T] 
for some T > 0, and that the initial state x(0) = xq is 
given for some xq £ R n . The input control of this system, 
v(t), is discrete since V is a finite set, and we assume that 
the function v(t) changes its values a finite number of times 
during the horizon interval [0, T], 

Such systems have been investigated in the past several 
years due to their relevance in control applications such as 
mobile robotics [7], vehicle control [19], switching circuits 
[1] and references therein, telecommunications [14], [9], and 
situations where a controller has to switch its attention among 
multiple subsystems [10] or data sources [5]. Of a particular 
interest in these applications is an optimal control problem 
where it is desirable to minimize a cost functional (criterion) 
of the form 

rT 
■J ;= / L(x)dt (2) 



associated with the various modes of the system^ Several 
variants of the maximum principle were derived for this 
problem in [18], [11], [15], and subsequently provably- 
convergent optimization algorithms were developed in [20], 
[15], [17], [8], [2]. We point out that two kinds of problems 
were considered: those where the sequence of modes is fixed 
and the controlled variable consists of the switching times 
between them, and those where the controlled variable is 
comprised of the sequence of modes as well as the switching 
times between them. We call the former problem the timing 
optimization problem, and the latter problem, the scheduling 
optimization problems. 

The timing optimization problem generally is simpler than 
the scheduling optimization problem since essentially it is a 
nonlinear-programming problem (albeit with a special struc- 
ture) having only continuous variables, while the scheduling 
problem has a discrete sequencing-variable as well. Fur- 
thermore, scheduling problems generally are NP hard, and 
computational techniques have to search for solutions that 
are suboptimal in a suitable sense. Thus, while the algorithms 
that were proposed early focused on the timing optimization 
problem, several different (and apparently complementary) 
approaches to the scheduling-optimization problem have 
emerged as well. Zoning algorithms that compute (itera- 
tively) the mode sequences based on geometric properties of 
the problem have been developed in [16], needle-variations 
techniques were presented in [3], and relaxation methods 
were proposed in [6], In contrast, the algorithm considered 
in this paper computes its iterations directly in the schedule 
space without resorting to relaxations, and as argued later in 
the sequel, may compute optimal (or suboptimal) schedules 
quite effectively. 

Our stating point is the algorithm we developed in [3] 
which alternates between the following two steps: (1). Given 
a sequence of modes, compute the switching times among 
them that minimize the functional J. (2). Update the mode- 
sequence by inserting to it a single mode at a (computed) 
time that would lead to the greatest-possible reduction rate 
in J. Then repeat Step 1, etc. 

The second step deserves some explanation. Fix a time 
t £ [0,T], and let us denote the system's mode at that time 
by M a . Now suppose that we replace this mode by another 
mode, denoted by Mp, over the time-interval [t.t + A] for 
some given A > 0, and denote by J(X) the cost functional 



for a given T > 0, where L : R n —$■ R is a cost function 
defined on the state trajectory. 

This general nonlinear optimal-control problem was for- 
mulated in [4], where the particular values of v £ V are 



The setting in [4] is more general since it involves a continuous-time 
control u 6 R k as well as a discrete control v. In this paper we focus 
only on the discrete control since it captures the salient points of switched- 
mode systems, and we defer discussion of the general case to a forthcoming 
publication. 



J defined by (2) as a function of A. We call the one-sided 
derivative ^Jt(O) the insertion gradient, and we note that if 
:ttt(0) < then inserting Mp for a brief amount of time at 
time t would result in a decrease in J, while if ^r(O) > 
then such an insertion would result in an increase in J. 
Now the second step of the algorithm computes the time 
t € [0, T] and mode Mg that minimize the insertion gradient, 
and it performs the insertion accordingly. We mention that 
if the insertion gradient is non-negative for every mode Mp 
and time t £ [0, T] then the schedule in question satisfies a 
necessary optimality condition and no insertion is performed. 

The aforementioned algorithm has a peculiar feature in 
that it solves a timing optimization problem between con- 
secutive mode-insertions. This feature appears awkward and 
suggests that the algorithm can be quite inefficient, but it is 
required for the convergence-proof derived in [3]. In fact, that 
proof breaks down if the insertions are made for schedules 
that do not necessarily comprise solution points of the timing 
optimization problem for their given mode-sequences. The 
reason seems to be in the fact that the insertion gradient is 
not continuous in the time-points at which the insertion of 
a given mode Mp is made. However, this lack of continuity 
can be overcome by other properties of the problem at hand, 
and this leads to the development of the algorithm that is 
proposed in this paper, which appears to be more efficient 
than the one in [3]. 

The algorithm we describe here computes its iterations 
directly in the space of mode-schedules without having to 
solve any timing optimization problems. Furthermore, at 
each iteration it switches the mode not at a finite set of times, 
but at sets comprised of unions of positive-length intervals in 
the time-horizon [0,T], The algorithm is based on the idea 
of the Armijo step size used in gradient-descent techniques 
[13], and it uses the Lebesgue measure of sets where the 
modes are to be changed as the step-size parameter. To the 
best of our knowledge this idea has not been used in extant 
algorithms for optimal control problems, and while it appears 
natural in the setting of switched-node systems, it may have 
extensions to other optimal-control settings as well. We prove 
the algorithm's convergence and its convergence-rate, which 
we show to be independent of the number of intervals where 
the modes are changed at a given iteration. 

The rest of the paper is organized as follows. Section 
II sets the mathematical formulation of the problem and 
recounts some established results. Section III carries out the 
analysis, while Section IV presents a simulation example. 
Finally, Section V concludes the paper. 

II. Problem Formulation and Survey of Relevant 
Results 

Consider the state equation (1) and recall that the initial 
state xo and the final time T > are given. We make the 
following assumption regarding the vector field f(x,v) and 
the state trajectory {x(t)}. 

Assumption 1: (i). For every v £ V, the function f(x,v) 
is twice-continuously differentiable (C 2 ) throughout R n . (ii). 
The state trajectory x{t) is continuous at all t £ [0,T]. 



Every mode-schedule is associated with an input control 
function v : [0, T] — > V, and we define an admissible mode 
schedule to be a schedule whose associated control function 
v(-) changes its values a finite number of times throughout 
the interval t £ [0,T]. We denote the space of admissible 
schedules by S, and a typical admissible schedule by a £ S. 
Given a £ E, we define the length of a as the number 
of consecutive different values of v on the horizon interval 
[0, T], and denote it by 1(a). Furthermore, we denote the ith 
successive value of v in a by v l , i = 1, . . . ,t(a), and the 
switching time between v l and v l+1 will be denoted by Tj. 
Further defining To := and T£/ a ) = T, we observe that the 
input control function is defined by v(t) = ViVi £ [r»_i, t»), 
i = 1, . . . ,£(a). We require that £(a) < oo but impose no 
upper bound on 1(a). 

Given a £ S, define the costate p £ R n by the following 
differential equation, 



P 



Xd-x M ) P 



(tx^ 



(3) 



with the boundary condition p(T) = 0. Fix time s £ [0, T), 
w £ V, and A > 0, and consider replacing the value of v(t) 
by w for every t £ [s, s + A). This amounts to changing 
the mode-sequence a by inserting the mode associated with 
w throughout the interval [s,s + A). Denoting by J(A) the 
value of the cost functional resulting from this insertion, 
the insertion gradient is defined by ^rqr(O). Of course this 
insertion gradient depends on the mode-schedule a, the 
inserted mode associated with w £ V, and the insertion time 
s, and hence we denote it by D aySW . We have the following 
result (e.g., [8]): 



D a 



= p( S ) T (f(x( S ),w)~f(x(s),v( S ))). (4) 



As mentioned earlier, if D a>s>w < then inserting to a 
the mode associated with ic on a small interval starting 
at time s would reduce the cost functional. On the other 
hand, if D a ^ iW > for all w £ V and s £ [0,T] then 
we can think of a as satisfying a local optimality condition. 
Formally, define D ayS := min{D CT . st0 : w £ V}, and define 
D a := infjD^s : s £ [0,T]}. Observe that D CT , S , V ( S ) = 
since v(s) is associated with the same mode at time s and 
hence a is not modified, and consequently, by definition, 
D a . s < and D a < as well. The condition D a = is 
a natural first-order necessary optimality condition, and the 
purpose of the algorithm described below is to compute a 
mode-schedule a that satisfies it. 

Our algorithm is a descent method based on the principle 
of the Armijo step size. Given a schedule a £ S, it computes 
the next schedule, a ne xt, by changing the modes associated 
with points s £ [0, T] where D^g < 0. The main point of 
departure from existing algorithms (and especially those in 
[3]) is that the set of such points s is not finite or discrete, but 
has a positive Lebesgue measure. Moreover, the Lebesgue 
measure of this set acts as the parameter for the Armijo 
procedure. 

Now one of the basic requirements of algorithms in 
the general setting of nonlinear programming is that every 



accumulation point of a computed sequence of iteration 
points satisfies a certain optimality condition, like stationarity 
or the Kuhn-Tucker condition. However, in our case such 
a convergence property is meaningless since the schedule- 
space S is neither finite dimensional nor complete, the latter 
due to the requirement that 1(a) < oo V a £ E, Conse- 
quently convergence of our algorithm has to be characterized 
by other means, and to this end we use Polak's concept of 
minimizing sequences [12]. Accordingly, the quantity D a 
acts as an optimality function [13], namely the optimality 
condition in question is D a = 0, while \D a \ indicates an 
extent to which a fails to satisfy that optimality condition. 
Convergence of an algorithm means that, if it computes a 
sequence of schedules {crk}^ =1 then, 



lim sup D a 

k— >oo 



0: 



(5) 



in some cases the stronger condition lirrifc^oo D ak = 
applies. In either case, for every e > the algorithm yields 
an admissible mode-schedule a G X satisfying the inequality 
D a > — e. Our analysis will yield Equation (5) by proving 
a uniformly-linear convergence rate of the algorithmic 

Since the Armijo step-size technique will play a key role 
in our algorithm, we conclude this section with a recount of 
its main features. Consider the general setting of nonlinear 
programming where it is desirable to minimize a C 2 function 
/ : R n — > R, and suppose that the Hessian -£s(x) is 
bounded on R n . Given x £ R n , a steepest descent from 
x is any vector in the direction —Vf(x); we normalize the 
gradient by defining h(x) := iivfeSlh anc ' ca ^ ~h(x) the 
steepest-descent direction. Let X(x) > denote the step size 
so that the next point computed by the algorithm, denoted 
by x nex t, is defined as 



Xnext = X — X(x)h(x). 



(6) 



The Armijo step size procedure defines X(x) by an approx- 
imate line minimization in the following way (see [13]): 
Given constants a £ (0, 1) and (3 £ (0, 1), define the integer 

j{x) by 

j(x) : min jj =0,1,..., : 

f(x - P j Vf(x)) - f(x) < -«^'||V/(x)|| 2 }, (7) 

and define 

X(x) = £ i(x) ||V/(:r)||. (8) 

Now the steepest descent algorithm with Armijo step size 
computes a sequence of iteration points Xk, k = 1,2,..., 
by the formula Xk+i = Xk — X(Xk)h(xk)', X(xk) is called 
the Armijo step size at Xk- The main convergence property 

"The reason for the "limsup" in (5) instead of the stronger form of 
convergence (with "lim" instead of "limsup") is due to technical peculiarities 
of the optimality function D„ that will be discussed later. We will argue that 
the stronger form of convergence applies except for pathological situations. 
Furthermore, we will define an alternative optimality function and prove 
the stronger form of convergence for it. The choice of the most-suitable 
optimality function is largely theoretical and will not be addressed in this 
paper. 



of this algorithm [13] is that every accumulation point x of 
a computed sequence \xk\^=\ satisfies the stationarity con- 
dition V/(x) = 0. Several results concerning convergence 
rate have been derived as well, and the one of interest to 
us is given by Proposition 1 below. Its proof is contained in 
the arguments of the proof of Theorem 1.3.7 and especially 
Equation (8b) in [13], but since we have not seen the result 
stated in the same way as in Proposition 1, we provide a 
brief proof in the appendix. 

Proposition 1: Suppose that fix) is C 2 , and that there 
exists a constant L > such that, for every x £ R n , 
||iJ(x)|| < L, where H(x) := §^(x). Then the following 
two statements are true: (1). For every x £ R n and for every 
A > such that A < f (1 - a)||V/(«)||, 



f(x-Xh(x))~f(x) < -aX\\Vf(x)\\. 



(2). For every x £ R n , 



X(x) > -0(l-a)||V/(aO||. 



(9) 



(10) 



This implies the following convergence result: 

Corollary 1: (1). There exists c > such that \/x £ R n , 



f(x next )-f(x) < -c||V/(z) 



(11) 



(2). If the algorithm computes a bounded sequence {xk} k *^ 1 
then 

lim Vf(x k ) = 0. (12) 

k— >-oo 

Proof: (1). Define c := f a(l-a)/3. Then (11) follows 
directly from Equations (9) and (10). 

(2). Follows immediately from part (1) and the fact that 
the sequence {f(xk)}kLi i s monotone non-increasing. ■ 

III. Algorithm for Mode-Scheduling 
Minimization 

To simplify the notation and analysis we assume first that 
the set V consists only of two elements, namely the system 
is bi-modal. This assumption incurs no significant loss of 
generality, and at the end of this section we will point out an 
extension to the general case where V consists of an arbitrary 
finite number of points. Let us denote the two elements of V 
by vi and V2- A mode-schedule a alternates between these 
two points, and we denote by {v 1 , . . . , tA" 7 )} the sequence 
of values of v associated with the mode-sequence comprising 
a. Denoting by v c the complement of v, we have that v t+1 — 
(V) c for all i= 1,..., t(0)- 1. 

Consider a mode-schedule a £ I] that does not satisfy the 
necessary optimality condition, namely D a < 0. Define the 
set S a fl as S a fi '■= {s £ [0, T] : D a ^ s < 0}, and note 
that 5,7,0 7^ 0. Recall that v(s) denotes the value of v at the 
time s. Then for every s £ 5<7,o which is not a switching 
time, an insertion of the complementary mode v(s) c at s 
for a small-enough period would result in a decrease of J. 
Our goal is to flip the modes (namely, to switch them to 
their complementary ones) in a large subset of 5 CT ,o that 
would result in a substantial decrease in J, where by the 



term "substantial decrease" we mean a decrease by at least 
aD^. for some constant a > 0. This "sufficient descent" in 
J is akin to the descent property of the Armijo step size as 
reflected in Equation (11). 

This sufficient-descent property cannot be guaranteed by 
flipping the mode at every time s € S a ,o- Instead, we search 
for a subset of 5 (Ti o where, flipping the mode at every s in 
that subset would guarantee a sufficient descent. This subset 
will consist os points s where D as is "more negative" than 
at typical points s e S„fi. Fix r\ <G (0, 1) and define the set 
Sa,r, by 



S a>v = {se [0,T] : D a , s <r)D a }. 



(13) 



Obviously 5 CTi7) ^ since D a < 0. Let n(S a ^ n ) denote 
the Lebesgue measure of S atV , and more generally, let /i(-) 
denote the Lebesgue measure on R. For every subset 5 C 
S a ^, consider flipping the mode at every point s E S, 
and denote by <r(5) the resulting mode-schedule. In the 
forthcoming we will search for a set S C S a ,ri that will 
give us the desired sufficient descent. 

Fix T) e (0,1). Let S : [0, M 5 ^)] -► 2 5 -" (the latter 
object is the set of subsets of S a , v ) be a mapping having the 
following two properties: (i) VA e [0,n(S a>rt )], 5(A) is the 
finite union of closed intervals; and (ii) VA e [0,/i(S , (Ti?? )], 
n(S(X)) = A. We define c(A) to be the mode-schedule 
obtained from a by flipping the mode at every time-point 
s G S(X). For example, VA <G [0, /x(5 CTi7 ,)] define s(X) := 
inf{s e 5 CTi7) : ^{[0,s] n S a . n ) = A}, and define 5(A) := 
[0, s(X)] n S atV . Then cr(X) is the schedule obtained from a 
by flipping the modes lying in the leftmost subset of S atT) 
having Lebesgue-measure A, and it is the finite union of 
closed intervals if so is S a . v . 

We next use such a mapping 5(A) to define an Armijo 
step-size procedure for computing a schedule o next from a. 
Given constants a <G (0, 1) and f3 e (0, 1), in addition to 
1] <G (0, 1). Consider a given a € S such that D a < 0. For 
every j = 0,1,.. ., define Xj := ft p(S a v ), and define j(a) 

by' 

j(a) := minjj = 0,1,..., : J(<j(Aj))- J(ct) < aXjDA. 

(14) 
Finally, define A(<r) := Aj( CT ), and set CT nex t := u(A(cr)). 

Observe that the Armijo step-size procedure is applied 
here not to the steepest descent (which is not defined in 
our problem setting) but to a descent direction defined by 
a Gateaux derivative of J with respect to a subset of the 
interval [0,T] where the modes are to be flipped. Generally 
this Gateux derivative is not necessarily continuous in A 
and hence the standard arguments for sufficient descent do 
not apply. However, the problem has a special structure 
guaranteeing sufficient descent and the algorithm's conver- 
gence in the sense of minimizing sequences. Furthermore, 
the sufficient descent property depends on n{S a ^) but is 
independent of both the string size £(a) and the particular 
choice of the mapping S : [0,^(5 CT .^)] —> 2 s °< r >. This 
guarantees that the convergence rate of the algorithm is not 



reduced when the string lengths of the schedules computed 
in successive iterations grows unboundedly. 

We next present the algorithm formally. Given constants 
a e (0,1), (3 e (0,1), and -q E (0,1). Suppose that for 
every a e S such that D a < there exists a mapping 
5 : [0, n(S„ tV )} — > 2 s "' 11 with the aforementioned properties. 

Algorithm 1: Step 0: Start with an arbitrary schedule <to <G 
S. Set k = 0. 

Step 1: Compute D ak . If D (7k = 0, stop and exit; otherwise, 
continue. 

Step 2: Compute S ak ^ as defined in (13), namely 5 CTfc .^ = 
{sE[0,T} : D ah , a < T]D ak }. 
Step 3: Compute j(o~k) as defined by (14), namely 

j(o-fc) = 
min {j =0,1,..., : J(<Jk(\j))-J(<Jk)<a\jD 0h }{\S) 

with Xj := ft n(S akiV ), and set X(<r k ) := X o{lJk) . 
Step 4: Define a k +i ■= (Jk{X{<Jk)), namely the schedule 
obtained from Uk by flipping the mode at every time-point 
s e 5(A(cr fe )). Set k = k + 1, and go to Step 1. 

It must be mentioned that the computation of the set 5 CTfc , v 
at Step 2 typically requires an adequate approximation. This 
paper analyzes the algorithm under the assumption of an 
exact computation of S ak . v , while the case involving adaptive 
precision will be treated in a later, more comprehensive 
publication. 

The forthcoming analysis is carried out under Assumption 
1, above. It requires the following two preliminary results, 
whose proofs follow as corollaries from established results 
on sensitivity analysis of solutions to differential equations 
[13], and hence are relegated to the appendix. Given a G 
S, consider an interval / := [si,S2] C [0,T] of a positive 
length, such that the modes associated with all s € I are 
the same, i.e., v(s) = v(s\) Vs e /. Denote by <7 Sl (7) the 
mode-sequence obtained from a by flipping the modes at 
every time s € [si,si + 7], and consider the resulting cost 
function J(a Sl (7)) as a function of 7 e [0, S2 — Si]. 

Lemma 1: There exists a constant K > such that, for 
every a <G S, and for every interval I — [s\, S2} as above, the 
function J(a Sl (-)) is twice-continuously differentiable (C 2 ) 
on the interval 7 e [0, S2 — S1]; and for every 7 e [0, S2 — si], 
\J(°~si (7)) I < K ("prime" indicates derivative with respect 
to 7). 

Proof: Please see the appendix. ■ 

We remark that the C 2 property of J(<t Si (-)) is in force 
only as long as v(s) = v(si) Vs e [si,S2]. The second 
assertion of the above lemma does not quite follow from the 
first one; the bound K is independent of the specific interval 
[si,s 2 \. 

Lemma 1 in conjunction with Corollary 1 (above) can 
yield sufficient descent only in a local sense, as long as the 
same mode is scheduled according to a. At mode-switching 
times D&S is no longer continuous in s, and hence Lemma 
1 cannot be extended to intervals where v(-) does not have 



a constant value. Nonetheless we can prove the sufficient- 
descent property in a more global sense with the aid of the 
following result, whose validity is due to the special structure 
of the problem. 

Lemma 2: There exists a constant K > such that for 
every a G S, for every interval I = [si,S2] as above (i.e., 
such that a has the same mode throughout I), for every 
7 G [0, 82 — si), and for every s > S2, 



\D 



<7«i(7).» 



D n 



< K-y. 



(16) 



Proof: Please see the appendix. ■ 

To explain this result, recall that a Sl (7) is the mode- 
schedule obtained from a by flipping all the modes on 
the interval [si,si + 7]. Thus, Equation (16) provides an 
upper bound on the magnitude of the difference between the 
insertion gradients of the sequences a and c Sl ( 7 ) at the same 
point s. Furthermore, Lemma 2 implies a uniform Lipschitz 
continuity of the insertion gradient at every point s > S2 
with respect to the length of the insertion interval 7. This is 
not the same as continuity of D as with respect to s, which 
we know is not true. 

Recall the following terminology: given aeS and S C 
[0, T], a(S) denotes the schedule obtained by flipping the 
mode of a at every r G S. 

Corollary 2: There exists K > such that, for every a G 
S, for every subset S C [0, T] comprised of a finite number 
of intervals, and for every s > sup{s G S}, 



\D, 



<r(S), 



Ar,a| < Kfi(S). 



(17) 



Proof: Let K > be the constant given by Lemma 2. 
Fix a G S, a subset S C [0, T] comprised of a finite number 
of intervals, and s > sup{s G S}. We can assume without 
loss of generality that each one of the intervals comprising S 
contains its lower-boundary point but not its upper-boundary 
point. Denote these intervals by Ij := [sij,S2,j), j = 
l,...,m for some m > 1, so that S — WJL^sij, S2.j)- 
Furthermore, by subdividing these intervals if necessary, we 
can assume that v(t) has a constant value throughout each 
interval Ij, namely all the modes in Ij are the same according 
to a. Note that these intervals need not be contiguous, i.e., it 
is possible to have Si,j+i > S2,j for some j — 1, . . . , m — 1. 
Define Sj := Uj =1 /j, j = 1, . . . , m, and note that 5 = 
S^. Furthermore, ju(S') = X)^Li( s 2.j — s i.j)- Next, we have 
that 



D, 



a{S), 



D n 



D*{Si),, - D*,s + 2^{D* {Si ),s - D^ Sj _ l)tB ). (18) 

i=2 

By Lemma 2, \D a{Sl ^ s -D a ^\ < K(s2,i-si,i), and for ev- 
ery j = 2,...,m, |-D« 7 (s ; ,), s --D<7(S : ,_i), s | < K(s2,j-,sij). 
By (16) |-D CT (S), S - D^ s \ < if X)JLi( s 2j - sij), and since 
/"(5 1 ) = Ejli( s 2j - «i,j), (17) follows. ■ 

We now can state the algorithm's property of sufficient 
descent. 

Proposition 2: Fix 77 G (0, 1), /3 e (0, 1), and a e (0,r]). 
There exists a constant c > such that, for every a E S 



satisfying Z^ < 0, and for every A S [0,/i(S' o -^)] such that 
A<c|£> ff |, 

J(cr(A)) - J(cr) < a\D a . (19) 

Proof: Consider cr <G S and an interval 7 := [si,S2) 
such that cr has the same mode throughout /. By Lemma 1, 
J(cr Sl (7)) is C 2 in 7 e [0, *2-*i), and by (4), J(cr Sl (0))' - 
D„ iSl . Fix a G (-,1). Suppose that D a , Sl < 0. By 
Proposition 1 (Equation (9)) there exists £ > such that, 
for every 7 > satisfying 7 < min{— £Da. Sl , S2 — Si}, 

J(cr Sl ( 7 ))- J(cr) < a 7 -D CT>Sl . (20) 

Furthermore, ^ does not depend on the mode-schedule a or 
on the interval I. 

Next, by Corollary 2 there exists a constant K > such 
that, for every a G S, for every set S 1 C [0, T] consisting of 
the finite union of intervals, and for every point s > sup{,5 G 
S}, 

\D a(Shs - D a . s \ < Kn(S). (21) 



Fix c > such that 



c < mm{—-(aT]-a),— }; 



aK 



K> 



(22) 



we next prove the assertion of the proposition for this c. 
Fix a G S such that _D CT < 0, and consider a set 5 C 
S CT .,j consisting of the finite union of disjoint intervals. By 
subdividing these intervals if necessary we can ensure that 
the length of each one of them is less than —£r)D a . Denote 
these intervals by Ij :— [sij, S2,j), j = 1, • ■ • , m (for some 
m > 0), define -jj := s 2j — sij, and define A := 2~2j=i Ij- 
Since Sij G S a ^ we have that D a , Sl . < —r\D a , and we 
recall that -/j < —^-qD^ Vj = 1, . . . , m. 

Next, we define the mode-schedules cr- 7 , j — 1, . . . , m, in 
the following recursive manner. For j = 1, cr 1 = cr^j ^71); 
and for every j — 2,...,m, a^ :— o' :, s ~ 1 (jj). In words, 
a 1 is obtained from a by flipping the mode at every time 
s G I\\ and for every j — 2, . . . , m, cr^ is obtained from 
cr^ 1 by flipping the mode at every time s G Ij. Observe 
that ai is also obtained from a by flipping the mode at 
every time s G Uj =1 ij. In particular, ct ? ™ is obtained from 
cr by flipping the modes at every time s G S. Since by 
assumption fi(S) = X)i=i 7j = ^> we W 1H use tne notation 
CT (S) := cr(A). ' 

Suppose that A < —cD a ; we next establish Equation (19), 
and this will complete the proof. Consider the difference- 
term JiyO'-') — J (a) for j = 1, . . . ,m. For j = 1, J (a 1 ) — 
J (a) < ajiD atSlj (by (20)); and since sij G S atV , 
D(t,si < r]D a , and hence 



J^ 1 )-^) < a llV D a 



(23) 



Nest, consider j = 2, . . . , m. An inequality like (23) does 
not necessarily hold since ct j is obtained from a by flipping 
the mode at every s G U^ =1 ij and //(U^ij) may be larger 
than —£ > D ( j :Sl , and therefore an inequality like (20) cannot 
be applied. A different argument is needed. 



Consider the term J(a^) — J(a). Subtracting and adding 
J(<7 : ' _1 ) we obtain, 

J(a j )-J(a) = J(a j )- J(a j - 1 ) + J^" 1 )- J(a). (24) 

Now er- 7 is obtained from er- 7-1 by flipping the mode at 
every time s e Ij and hence cr- 7 = cr^~ 1 (7j), while cr- 7-1 = 
(T^~ 1 (0) since in the latter term no mode is being flipped. 
Therefore, 

Jtf) - Jio*- 1 ) = J«;j(l 3 ))-J(ai;j(0)). (25) 

We next show that D a j-i si < in order to be able to use 
Equation (20). By (17), 

\D aJ -i iSl .-D a , Sl J < KTPr^. (26) 

By definition S^ ji < E"=i 7» = ^> by assumption 
A < c|£> CT |; and by (22) K < 2 ; consequently, and by (26), 
IAiJ-Ssi,., - Ar.siJ < v\ d <t\- But s i,j ^ S ayT) and hence 
■Do-.si < V^a, and this implies that D a j-i Sl < 0. 
An application of (20) to (25) now yields that 

J(a j ) - Jtf- 1 ) < a-YjD^-i^. (27) 

We do not know whether or not D a j-i_ Sl < r]D a , but we 
know that D„ tSl < -qD a (since sij € S aiV ). Applying (26) 
to (27) we obtain that 

J(a j ) - J^- 1 ) < 07,-^-i, aiJ = 

D„.. .) < 



a 1] D a, SlJ + a l]( D <^- 1 ,si, j 



J-l 



WyjDv,*!,! + a l3 K ^2^- 



(28) 



But -D CT . Sl < 77-Do- (since sij € S^,,), and hence, J{cr^) — 
J{a^~ x ) < ajji]D a + aKjj YH=i !%• Using this inequality 
in (24) yields the following one, 



J(a J )-J(cr) < 07^77^ + 0^7^ 7i + J( ( j j - 1 )-J(ct). 

(29) 
Apply (29) repeatedly and recursively with j = 1 , . . . , m to 
obtain, after some algebra, the following inequality: 

J(a m )-J(a) < aiJ^lihDv + aK ^ 77,. (30) 



But E™i7* = A, Em=m^7*7£ < |(lXi7i 
and a m := cr(A), and hence, 



.A 2 , 



1 



J(cr(A)) - J(cr) < aXr]D a + -aKX 2 . (31) 

By assumption A < —cD a , and by (22) aKc < 2(ai] — a), 
and this, together with (31), implies (19). The proof is now 
complete. ■ 

General results concerning sufficient descent, analogous to 
Proposition 2, provide key arguments in proving asymptotic 
convergence of nonlinear-programming algorithms (see, e.g., 
[13]). In our case, the optimality function has the peculiar 
property that it is discontinuous in the Lebesgue measure 
of the set where a mode is flipped. To see this, recall 



that Ar,«,t;(«) =P( s ) T (f( x ( s )> v ( s ) c ) ~ f( x ( s ), v ( s ))) ( see 
Equation (4)), and hence a change of the mode at time s 

would flip the sign of D a>BtV r B \. This can result in situations 
where \D a \ is "large" while S a . v is "small", and for this rea- 
son, convergence of Algorithm 1 is characterize by Equation 
(5) with the limsup rather than with the stronger assertion 
with lira. This is the subject of the following result. 

Corollary 3: Suppose that Algorithm 1 computes a se- 
quence of schedules, {o~k}'j?= 1 - Then Equation (5) is in force, 
namely lim sup^,^^ D ak = 0. 

Proof: Suppose, for the sake of contradiction, that 
Equation (5) does not hold. Then the sufficient-descent prop- 
erty proved in Proposition 2 implies that lim^oo A(<7&) = 0, 
for otherwise Equation (19) would yield lim^oo J{o~k) = 
—00 which is impossible. Next, By the definition of A(<7fc) 
(Step 3 of the algorithm), there exists fco such that Vfc > k , 
A((T fe ) = /j,(S aktV ), and hence lim/^oo /j,(S ( , ktV ) = 0. In this 
case, <7fc+i is obtained from u^ by flipping the modes at 
every s € S akyT} . By the perturbation theory of differential 
equations (e.g., Proposition 5.6.7 in [13]), x and p are 
Lipschitz continuous in their L°° norms with respect to the 
Lebesgue measure of the sets where the modes are flipped, 
i.e. /j,(S aktV ). Therefore, and by (4) and the definition of 
S aktV , there exist k\ > fco and C, € (0, 1) such that Vfc > fei, 



D 



<Tk + l,W 



> CD, 



&k,V> 



implying that lirrifc 



*-'<Tk, , n ~ 



However this is a contradiction to the assumption that (5) 
does not hold, thus completing the proof. ■ 

Alternative optimality functions can be considered as well, 
like the term D a n(S a . v ), where it is apparent Equation (19) 
that lirrife^oo D„ k n(S akiV ) = 0. The choice of the "most 
appropriate" optimality function is an interesting theoretical 
question that will be addressed elsewhere, while here we 
consider the simplest and (in our opinion) most intuitive 
optimality function D a , despite its technical peculiarities. 

Finally, a word must be said about the general case where 
the set V consists of more than two points. The algorithm 
and much of its analysis remain unchanged, except that for 
a given a <G S, at a time s, the mode associated with v(s) 
should be switched to the mode associated with the point 
w E V that minimizes the term D UySW . 

IV. Numerical Example 

We tested the algorithm on the double-tank system shown 
in Figure 1. The input to the system, v, is the inflow rate 
to the upper tank, controlled by the valve and having two 
possible values, v\ = 1 and v-i = 2. x\ and x^ are the 
fluid levels at the upper tank and lower tank, respectively, as 
shown in the figure. According to Toricelli's law, the state 
equation is 



Xl 
XI 



v - 



(32) 



with the (chosen) initial condition £i(0) = 3:2(0) = 2.0. 
Notice that both x\ and x^ must satisfy the inequalities 1 < 
Xi < 4, and if v — 1 indefinitely than lim^oo x. h = 1, while 
if v = 2 indefinitely then lim^oo Xi(t) = 4, i = 1, 2. 







X| 



*2 



Fig. 1. Two-tank system 
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Fig. 2. Control (schedule) obtained after 100 iterations 



The objective of the optimization problem is to have the 
fluid level in the lower tank track the given value of 3.0, and 
hence we chose the performance criterion to be 



J = 2 



(X2 



3) 2 di, 



(33) 



for the final-time T — 20. The various integrations were 
computed by the forward-Euler method with At = 0.01. For 
the algorithm we chose the parameter- values a — (3 = 0.5 
and r\ = 0.6, and we ran it from the initial mode-schedule 
associated with the control input v(i) = 1 V t € [0, 10] and 
u(*)=2V*e(10,20]. 

Results of a typical run, consisting of 100 iterations of 
the algorithm, are shown in Figures 2-5. Figure 2 shows 
the control computed after 100 iterations, namely the input 
control v associated with cioo- The graph is not surprising, 
since we expect the optimal control initially to consist of 
v = 2 so that X2 can rise to a value close to 3, and then to 
enter a sliding mode in order for X2 to maintain its proximity 
to 3. This is evident from Figure 2, where the sliding mode 
has begun to be constructed. Figure 3 shows the resulting 
state trajectories x\(t) and X2{t), t € [0,T], associated with 
the last-computed schedule uioo- The jagged curve is of x\ 
while the smoother curve is of x%. It is evident that X2 climbs 
towards 3 initially and tends to stay there thereafter. Figure 
4 shows the graph of the cost criterion J(<Jk) as a function 
of the iteration count k = 1, . . . , 100. The initial schedule, 
o\, is far away from the minimum and its associated cost is 
J(cri) = 70.90, and the cost of the last-computed schedule is 
■^(cioo) = 4.87. Note that J(cfc) goes down to under 8 after 
3 iterations. Figure 5 shows the optimality function D ah as 



a function of the iteration count k. Initially D a 



-14.92 



while at the last-computed schedule D aum = —0.23, and it 
is seen that D ak makes significant climbs towards in few 
iterations. We also ran the algorithm for 200 iterations from 
the same initial schedule <n, in order to verify that J(<Jk) 
and D ak stabilize. Indeed they do, and J declined from 
J(&ioo) = 4.87 to J(<72oo) = 4.78, while the optimality 
functions continues to rise towards 0, from D aum = —0.23 
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Fig. 3. x\ and X2 vs. t 



V. Conclusions 

This paper proposes a new algorithm for the optimal 
mode-scheduling problem, where it is desirable to minimize 
an integral-cost criterion defined on the system's state trajec- 
tory as a function of the modes' schedule. The algorithm is 
based on the principle of gradient descent with Armijo step 
sizes, comprised of the Lebesgue measures of sets where the 
modes are being changed. Asymptotic convergence is proved 
in the sense of minimizing sequences, and simulation results 
support the theoretical developments. Future research will 
refine the proposed algorithmic framework and apply it to 
large-scale problems. 

VI. Appendix 

The purpose of this appendix is to provide proofs to 
Proposition 1, and Lemmas 1 and 2. 

Proof of Proposition 1. 

(1). The main argument is based on the following form of 
the second-order Taylor series expansion: For every x € R n 
and y € R n , 

f(x + y)-f(x) = (Vf(x),y) + 



(l-0(H(x + tiy)y,y)dt;, 



(34) 



to D n 



-0.062. 



where (•) denotes inner product in R n . Apply this with y 
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Fig. 4. Cost criterion vs. iteration count 
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Fig. 5. Optimality function vs. iteration count 



-Xh(x) to obtain, 

f(x-Xh(x))-f(x) 



-\(Vf(x),h(x)) 



A 2 / (l-£)(H(x-Z\h(x))h(x),h(x))d£. (35) 
Jo 

Add aA||V/(x)|| to both sides of this equation, and use the 
fact that ||iJ(-)|| < L, to obtain (after some algebra) that 

/(x-A/i(x))-/(x) + aA||V/(x)|| 

-A((l-a)||V/(x)||-^). (36) 



< 



Now if < A < f (1 - a)||V/(x)|| then the Right-Hand 

side of (36) is non-positive, hence Equation (9) is satisfied. 

(2). Follows directly from Part (1), Equation (7), and the 

definition of A(x) (8). ■ 

The proofs of Lemma 1 and Lemma 2 follow as corollaries 
from established results on sensitivity analysis of solutions 
to differential equations, presented in Section 5.6 of [13]. In 
fact, the results of interest here involve mode-insertions via 
needle variations, which is a special case of the setting in 
[13] where general variations in the control are considered. 
Furthermore, the perturbations here are parameterized by a 
one-dimensional variable and hence the results are in terms 
of derivatives in the usual sense, while those in [13] are in 
terms of Gateaux or Frechet derivatives. 



Proof of Lemma 1. By Proposition 5.6.5 in [13] and 
the Bellman-Gronwall Lemma, the terms ||x(£)||z,oo are 
uniformly bounded over the space of controls v associated 
with every a G E, The costate equation (3) yields a 
similar result for |[p(t)||z,°°. Next, recall that v(-) has a 
constant value throughout the interval [si,S2J, and hence 
the differentiability assumptions of Theorem 5.6.10 in [13] 
are valid. This theorem implies that J(a Sl ('y)) exists and 
is expressed in terms of the Hamiltonian and its first two 
derivatives, hence it is uniformly bounded. ■ 

Proof of Lemma 2. Since v(-) has a constant value 
throughout the interval [si,S2], the assumptions made in 
the statement of Lemma 5.6.7 in [13] are in force. This 
implies a uniform Lipschitz continuity of x and p with 
respect to variations in 7. In the setting of Lemma 2, the 
needle variations is made at the same point s > S2 for both 
mode-schedules a and a Sl (7), and hence, and by Equation 
(4), Equation (16) follows. ■ 

References 

[1] S. Aimer, S. Mariefhoz, and M. Morari. Optimal Sampled Data Control 
of PWM Systems Using Piecewise Affine Approximations, Proc. 49th 
CDC, Atlanta, Georgia, December 15-17, 2010. 

[2] S.A. Attia, M. Alamir, and C. Canudas de Wit. Sub Optimal Control 
of Switched Nonlinear Systems Under Location and Switching Con- 
straints. Proc. 16th IFAC World Congress, Prague, the Czech Republic, 
July 3-8, 2005. 

[3] H. Axelsson, Y. Wardi, M. Egerstedt, and E. Verriest. A Gradient 
Descent Approach to Optimal Mode Scheduling in Hybrid Dynamical 
Systems. Journal of Optimization Theory and Applications, Vol. 136, 
pp. 167-186, 2008. 

[4] M.S. Branicky, V.S. Borkar, and S.K. Mitter. A Unified Framework 
for Hybrid Control: Model and Optimal Control Theory. IEEE Trans- 
actions on Automatic Control, Vol. 43, pp. 31-45, 1998. 

[5] R. Brockett. Stabilization of Motor Networks. IEEE Conference on 
Decision and Control, pp. 1484-1488, 1995. 

[6] T. Caldwell and T. Murphy. An Adjoint Method for Second-Order 
Switching Time Optimization. Proc. 49th CDC, Atlanta, Georgia, 
December 15-17, 2010. 

[7] M. Egerstedt. Behavior Based Robotics Using Hybrid Automata. 
Lecture Notes in Computer Science: Hybrid Systems III: Computation 
and Control, Springer Verlag, pp. 103-116, Pittsburgh, PA, March 
2000. 

[8] M. Egerstedt, Y. Wardi, and H. Axelsson. Transition-Time Optimiza- 
tion for Switched Systems. IEEE Transactions on Automatic Control, 
Vol. AC-51. No. 1, pp. 110-115, 2006. 

[9] D. Hristu-Varsakelis. Feedback Control Systems as Users of Shared 
Network: Communication Sequences that Guarantee Stability. IEEE 
Conference on Decision and Control, pp. 3631-3631, Orlando, FL, 
2001. 
[10] B. Lincoln and A. Rantzer. Optimizing Linear Systems Switching. 
IEEE Conference on Decision and Control, pp. 2063-2068, Orlando, 
FL, 2001. 
[11] B. Piccoli. Hybrid Systems and Optimal Control. Proc. IEEE Confer- 
ence on Decision and Control, Tampa, Florida, pp. 13-18, 1998. 
[12] E. Polak and Y Wardi. A Study of Minimizing Sequences. SIAM 
Journal on Control and Optimization, Vol. 22, No. 4, pp. 599-609, 
1984. 
[13] E. Polak. Optimization Algorithms and Consistent Approximations. 

Springer-Verlag, New York, New York, 1997. 
[14] H. Rehbinder and M. Sanfirdson. Scheduling of a Limited Commu- 
nication Channel for Optimal Control. IEEE Conference on Decision 
and Control, Sidney, Australia, Dec. 2000. 
[15] M.S. Shaikh and P. Caines. On Trajectory Optimization for Hybrid 
Systems: Theory and Algorithms for Fixed Schedules. IEEE Confer- 
ence on Decision and Control, Las Vegas, NV, Dec. 2002. 



[16] M.S. Shaikh and RE. Caines. Optimality Zone Algorithms for Hybrid 
Systems Computeation and Control: From Exponential to Linear Com- 
plexity. Proc. IEEE Conference on Decision and Control/European 
Control Conference, pp. 1403-1408, Seville, Spain, December 2005. 

[17] M.S. Shaikh and RE. Caines. On the Hybrid Optimal Control Problem: 
Theory and Algorithms. IEEE Trans. Automatic Control, Vol. 52, pp. 
1587-1603, 2007. 

[18] HJ. Sussmann. A Maximum Principle for Hybrid Optimal Control 
Problems. Proceedings of the 38th IEEE Conference on Decision and 
Control, pp. 425-430, Phoenix, AZ, Dec. 1999. 

[19] L.Y. Wang, A. Beydoun, J. Cook, J. Sun, and I. Kolmanovsky. Optimal 
Hybrid Control with Applications to Automotive Powertrain Systems. 
In Control Using Logic-Based Switching, Vol. 222 of LNCIS, pp. 190- 
200, Springer- Verlag, 1997. 

[20] X. Xu and P. Antsaklis. Optimal Control of Switched Autonomous 
Systems. IEEE Conference on Decision and Control, Las Vegas, NV, 
Dec. 2002. 



